Automatic pancreas ct segmentation method based on a saliency-aware densely connected dilated convolutional neural network

ABSTRACT

The present invention discloses an automatic pancreas CT segmentation method based on a saliency-aware densely connected dilated convolutional neural network. Under a coarse-to-fine two-step segmentation framework, the method uses a densely connected dilated convolutional neural network as a basis network architecture to obtain multi-scale image feature expression of the target. An initial segmentation probability map of the pancreas is predicted in the coarse segmentation stage. A saliency map is then calculated through saliency transformation based on a geodesic distance transformation. A saliency-aware module is introduced into the feature extraction layer of the densely connected dilated convolutional neural network, and the saliency-aware densely connected dilated convolutional neural network is constructed as the fine segmentation network model. A coarse segmentation model and the fine segmentation model are trained using a training set, respectively.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of International ApplicationNo. PCT/CN2021/073132, filed on Jan. 21, 2021, which claims priority toChinese Application No. 202010274508.6, filed on Apr. 9, 2020, thecontents of both of which are incorporated herein by reference in theirentireties.

TECHNICAL FIELD

The present invention belongs to a field of CT segmentation technology,and particularly relates to a pancreas CT automatic segmentation methodbased on a saliency densely connected dilated convolutional network.

BACKGROUND

In the diagnosis and prognosis prediction of pancreatic diseases, anecessary task is to identify and segment pancreatic regions frommedical images (such as magnetic resonance Imaging (MR), computertomography (CT)). Accurate segmentation of the pancreas can provide avariety of important biomarkers, such as volume, three-dimensional shapefeatures, three-dimensional curved surface features, and so on. Forexample, pancreatic volume can provide assistance for evaluatingendocrine and exocrine functions of the pancreas, and predicting theoccurrence of pancreatic fistula after pancreaticoduodenectomy. However,it is time-consuming and labor-intensive to delineate the pancreasboundary from the three-dimensional images layer by layer, and it isvariable under different operators and different operations. Therefore,the study of fast and robust pancreas automatic segmentation algorithmhas important clinical significance and value. Automatic Pancreas CTsegmentation is a challenging task because: 1) the pancreas occupies avery small proportion of entire CT, resulting in a huge gap between theforeground and the background of the image segmentation task; 2) thepancreas shares the gray-scale texture distribution with surroundingorgans and tissues, and the boundary is blurred or there is no visibleboundary; 3) the pancreas has an irregular shape, and there are greatdifferences in location and size. These characteristics make traditionalsegmentation algorithms based on image information (such as level set,graph cut, threshold method) not suitable for automatic pancreassegmentation.

In recent years, deep learning methods have made great breakthroughs inthe task of pancreas segmentation. Existing methods generally adopt atwo-step or iterative segmentation method from coarse to fine, that is,first a coarse segmentation network model is trained to predict aninitial segmentation area of the pancreas. Then, a region of interest ofthe pancreas image is cropped by using the position information of theinitial segmentation, to obtain smaller image blocks as the input of thefine segmentation network, and a fine segmentation network model istrained to obtain the final segmentation result of the pancreas.Although reducing the detection difficulty caused by the smallpancreatic area, the method does not make full use of the initialsegmentation information and only uses the position information. On theother hand, in the current pancreas segmentation methods based on deeplearning, the more commonly used network models are end-to-end U-Net,DenseNet, ResNet, etc., without special consideration of thecharacteristics of variable size of the pancreas.

SUMMARY

The purpose of the present invention is aiming at the shortcomings ofthe existing coarse-to-fine two-step deep convolutional neural networkpancreas CT segmentation method, and proposes a small target automaticsegmentation model based on a saliency-aware densely connected dilatedconvolutional neural network to realize the accurate segmentation ofpancreas from CT images.

The purpose of the present invention is achieved through the followingtechnical solutions: the present invention uses a densely connecteddilated convolutional neural network as the basic network architecture,and introduces a novel saliency-aware module based on a geodesicdistance transformation into a network, and constructs a saliency-awaredensely connected dilated convolutional neural network for precisesegmentation of the pancreas. Under a coarse-to-fine two-stepsegmentation framework, the coarse segmentation information is convertedinto saliency information through the saliency-aware module and added tothe feature extraction layer of the fine segmentation network. Thecoarse segmentation information is effectively used to guide the finesegmentation task and improve the final accuracy of pancreassegmentation. The constructed saliency-aware module does not have anyassumptions about the basic network architecture and can be extended toother network structures. At the same time, the densely connecteddilated convolutional neural network can obtain densely multi-scalefeature expression, effectively overcome the difficulty of variable sizeof the pancreas, and improve the segmentation accuracy of the networkmodel. The specific implementation steps of the present invention are asfollows:

(1) preprocessing of the training set, including:

collecting CT volume data and making a standard pancreas segmentationresult of the data;

denoting 3D CT volume as X, and slice number of the volume data is L;the corresponding standard segmentation is Y=(y_(j),j=1, . . . ,|X|),y_(j)={0,1}, wherein |X| represents the number of all voxels in X,y_(j)=1 or y_(j)=0 represents that voxel j belongs to the pancreas orbackground, respectively;

Slicing each volume X into two-dimensional image slices alone axialview, and combining three consecutive images into a three-channelpseudo-color image, denoted as X_(A,l)(l=1, . . . , L);

Slicing Y into two-dimensional image slices alone axial view, andcombining three consecutive label images into a three-channel labelimage, denoted as Y_(A,l)(l=1, . . . , L);

adjusting the contrast of each two-dimensional image X_(A,l);

cropping each pair of two-dimensional images X_(A,l) and Y_(A,l) intofixed size image block as input of a coarse segmentation network;

(2) coarse segmentation stage, including:

constructing a deep convolutional neural network for coarse segmentationfor pancreas CT, and training the network by training samples to obtaina pancreas initial segmentation model;

feeding the test image into the trained network model to assign eachpixel in the image a probability value P_(A,l) ^(C) of belonging to thepancreas; binarizing the probability value to obtain an initialsegmentation result Z_(l) ^(C) of the pancreas;

cropping the region of interest on the original input image according tothe initial segmentation result, and denoting the cropped image asX_(A,l) ^(F); Similarly, cropping the label image and denoting it asY_(A,l) ^(F);

(3) calculating a saliency map based on a geodesic distance, including:

calculating a geodesic distance map according to the original imageX_(A,l) ^(F) and the initial segmentation result Z_(l) ^(C);

performing a saliency transformation on the geodesic distance map toobtain a saliency map S(X_(A,l) ^(F));

(4) fine segmentation stage, including:

constructing a saliency-aware densely connected dilated convolutionalneural network: adding a saliency-aware module after each dense block ofthe densely connected dilated convolutional neural network to introducesaliency information based on the geodesic distance map. DenotingF(X_(A,l) ^(F)) as a output feature map of the dense block, and takingthe saliency map S(X_(A,l) ^(F)) as a weight map to act on the featuremap:

L _(c)(X _(A,l) ^(F))=F _(c)(X _(A,l) ^(F))⊗S(X _(A,l) ^(F)),

wherein c∈{1, 2, . . . , C} is the index of feature map channel, and ⊗represents element-wise multiplication;

combining the obtained L_(c)(X_(A,l) ^(F)) with the original feature mapF_(c)(X_(A,l) ^(F)) through an identity transformation and a parameter

H _(c)(X _(A,l) ^(F))=F _(c)(X _(A,l) ^(F))⊗η*L _(c)(X _(A,l) ^(F)),

where the symbol ⊗ represents element-wise addition, and the parameter ηis an adjustment coefficient, which is obtained through the networktraining. The output H_(c)(X_(A,l) ^(F)) of the obtained saliency-awaremodule is used as an input of next transition layer to participate inthe calculation;

feeding the original image X_(A,l) ^(F) and the corresponding labelimage Y_(A,l) ^(F) into the constructed saliency-aware densely connecteddilated convolutional neural network for parameter training, andobtaining the fine segmentation network model;

(5) fusing multi-layer two-dimensional pancreatic segmentation resultsto obtain a three-dimensional pancreatic segmentation result,

for a test image X^(test), slicing X^(test) along axial view plane toobtain a two-dimensional image sequence, and combining three consecutivegray-scale images into a three-channel pseudo-color image X_(A,l)^(test)(l=1, . . . , L); feeding each image into the trained coarsesegmentation model and the fine segmentation model successively, andobtaining a prediction result P_(A,l) for each image about pancreasarea;

performing multi-layer probability value fusion on the prediction resultP_(A,l), and a predicted segmentation result Z_(A)(l) of thetwo-dimensional original image of each layer is the average value ofpredicted values of three pseudo-color images in this layer.

Further, wherein the adjusting the contrast of each two-dimensionalimage X_(A,l), is specifically: the HU values of images are truncatedinto range [−100, 240], and then normalized to be in range [0, 1].

Further, in the coarse segmentation stage, the densely connected dilatedconvolutional neural network is configured to perform coarsesegmentation on pancreas CT, and the network is composed of two parts:

the first part is a feature extraction layer consists of a denselyconnected network 161 (DenseNet161), including aconvolution-normalization-ReLU activation-pooling layer, four denseblocks and four transition layers. The number of feature maps obtainedfrom the first dense block is 96, and a subsequent growth rate is 48;the size of the output feature maps of the feature extraction layer isof ⅛ input image size;

the second part is three densely connected dilated convolutional layers(atrous convolutional layer), dilation rates are 3, 6 and 12,respectively. The input of each dilated convolutional layer is theoutput of all previous dilated convolutional layers;

in the end of the network is an upper sampling layer with an uppersampling rate of 8 and a classification layer to predict the probabilityof belonging to pancreas region for each pixel in the image;

the loss function of the network is set as:

$E = {{- \frac{1}{n}}{\sum\limits_{j = 1}^{n}{\sum\limits_{c \in {\{{0,1}\}}}{{I\left( {y_{j} = c} \right)}\log\mspace{14mu}{p\left( {z_{j} = c} \right)}}}}}$

where, n is the number of pixels in the input image, y_(j) and z_(j) aretrue label and predicted label of pixel j, respectively, and c=1 or 0represents foreground or background, respectively. The function 1(·) isan characteristic function, and p(·) is a probability function predictedby the network model.

Further, in the coarse segmentation stage, feeding prepared training setinto the densely connected dilated convolutional neural network, andobtaining network parameters by the back-propagation algorithm.

Further, the geodesic distance map G(S_(f),X_(A,l) ^(F)) is calculatedas follows:

denoting sets of pixels belonging to the pancreas area and thebackground as S_(f) and S_(g), respectively according to the initialsegmentation Z_(l) ^(C). The geodesic distance G⁰(i,S_(f),X_(A,l) ^(F))from pixel i to S_(f) is defined as:

${{G^{0}\left( {i,S_{f},X_{A,l}^{F}} \right)} = {{{}_{j \in S_{f}}^{}{}_{}^{}}\left( {i,j,X_{A,l}^{F}} \right)}},{{D_{geo}\left( {i,j,X_{A,l}^{F}} \right)} = {\min\limits_{p \in {\mathcal{P}{({i,j})}}}{\int_{0}^{1}{{{{\nabla{X_{A,l}^{F}\left( {p(s)} \right)}} \cdot {u(s)}}}d\; s}}}},$

where

(i,j) is a set of all feasible paths from pixel i to j; path p isparameterized by s∈[0,1] as p(s);

${u(s)} = \frac{{p(s)}^{\prime}}{{p(s)}^{\prime}}$

is a unit vector that is tangent to the path direction, and p(s)′represents derivation for s. The image derivative ∇X_(A,l) ^(F)(p(s))requires the path from i to j to be the shortest in terms of imagegray-scale similarity. The symbol ∫₀ ¹ds represents an integral from 0to 1, the symbol ∥ ∥ represents l₁ norm, and the symbol representsbelongs to;

denoting the geodesic distance map as G(S_(f),X_(A,l) ^(F)), and thecalculation is as follows:

${G\left( {i,S_{f},X_{A,l}^{F}} \right)} = {1 - \frac{G^{0}\left( {i,S_{f},X_{A,l}^{F}} \right)}{\max\limits_{i}\;{G^{0}\left( {i,S_{f},X_{A,l}^{F}} \right)}}}$

where i is a pixel in the geodesic distance map.

Further, the saliency map S(X_(A,l) ^(F)) is calculated as follows:

S(X _(A,l) ^(F))=r(G(S _(f) ,X _(A,l) ^(F)))

where r(·) is a size-preserved saliency transformation that uses one 3×3convolutional layer.

Further, in step (5), the predicted segmentation result Z_(A)(l) of thetwo-dimensional original image of each layer is calculated as follows:

Z _(A)(l)=⅓(P _(A,l−1)(3)+P _(A,l)(2)+P_(A,l+1)(1)),

where P_(A,l)(i),i=1, 2, 3 respectively represent the P_(A,l) value ofi^(th) channel.

The present application also proposes a pancreas CT automaticsegmentation system based on a saliency-aware densely connected dilatedconvolutional neural network, including:

a memory for storing computer executable instructions; and

a processor for realizing the steps in the above method when executingthe computer executable instructions.

The present application also proposes a computer-readable storage mediumhaving stored therein computer executable instructions which. whenexecuted by a processor, implement the steps in the above method.

The beneficial effects of the present invention are: the presentinvention is based on a coarse-to-fine two-step segmentation framework,and uses a densely connected dilated convolutional neural network as thebasic network architecture for pancreas segmentation tasks, which cansolve the difficulty in detection and segmentation of pancreas underconditions of variable sizes and positions. At the same time, thesaliency-aware module based on a geodesic distance transformation isinnovatively introduced into the densely connected dilated convolutionalneural network, and the coarse segmentation result is effectivelyconverted into saliency information and added to the featurerepresentation layer of the fine segmentation network model to improvethe accuracy of the pancreas segmentation. The proposed saliency-awaremodule has good scalability and can be transplanted to other deepconvolutional network structures.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a flow chart of pancreas segmentation based on asaliency-aware densely connected dilated convolutional neural networkaccording to the present invention;

FIG. 2 is the architecture diagram of the densely connected dilatedconvolutional neural network;

FIG. 3 is an embodiment of segmentation of the present invention.

DESCRIPTION OF EMBODIMENTS

To make the objectives, technical solutions and advantages of thepresent invention more comprehensible, embodiments of the presentinvention is described in further detail below with reference to theaccompanying drawings.

Many specific details are described in the following description tofacilitate a full understanding of the present invention, but thepresent invention can also be implemented in other ways different fromthose described here. Those skilled in the art can make similarpromotion without violating the connotation of the present invention.Therefore, the present invention is not limited by the specificembodiments disclosed below.

The dilated convolution of the present application is: holes areinjected into a standard convolution kernel to increase the receptionfield of a model. HU value: that is, CT value, which is a measurementunit for measuring a density of a local tissue or organ of the humanbody, and it is usually called Hounsfield unit (HU), for example, air is−1000 and dense bone is +1000.

A coarse-to-fine two-step segmentation process based on a saliency-awaredensely connected dilated convolutional neural network proposed in thepresent application is shown in FIG. 1. The process is as follows: inthe coarse segmentation stage, firstly, a densely connected dilatedconvolutional neural network (as shown in FIG. 2) is constructed forcoarse segmentation of pancreatic region from three-dimensional pancreasCT volume, and geodesic distance transformation and saliencytransformation are performed on the coarse segmentation result to obtaina saliency map. At the same time, the coarse segmentation result is usedto crop a region of interest of input original image, and a croppedimage is used as an input of the fine segmentation model. In the finesegmentation stage, a saliency-aware densely connected dilatedconvolutional neural network model is constructed. The model uses theoriginal image and saliency information calculated from the coarsesegmented pancreatic region for training and predicting, to realizeaccurate segmentation of the pancreas. The specific steps are asfollows:

(1) preprocessing of the training set

(1.1) collecting CT volume data and making a standard pancreassegmentation result of the data; denoting 3D CT volume as X, whose sizeis 512×512×L, where L is the slice number of the volume data; thecorresponding standard segmentation is Y=(y_(j),j=1, . . . ,|X|),y_(j)={0,1}, where |X| represents the number of all voxels in X,y_(j)=1 or y_(j)=0 represents that voxel j belongs to the pancreas orbackground, respectively; slicing each volume X into two-dimensionalimage slices alone axial view and combining three consecutive imagesinto a three-channel pseudo-color image, denoting as X_(A,l)(l=1, . . ., L); similarly, slicing Y into two-dimensional image slices alone axialview, and combining three consecutive label images into a three-channellabel image, denoting as Y_(A,l)(l=1, . . . , L);

(1.2) adjusting the contrast of each two-dimensional image X_(A,l).Specifically, the HU values of images are truncated into range [−100,240], and then normalized to be in range [0, 1].

(1.3) cropping each pair of two-dimensional images X_(A,l) and Y_(A,l)into image block with size of 448×448, and take it as input of thecoarse segmentation network.

(2) The densely connected dilated convolutional neural network isconfigured to perform coarse segmentation on pancreas CT

(2.1) constructing a densely connected dilated convolutional neuralnetwork (as shown in FIG. 2), and the network is composed of two parts.First, a feature extraction layer of a densely connected network 161(DenseNet161) is used as the first part, which includes aconvolution-normalization-ReLU activation-pooling layer, four denseblocks and four transition layers; the number of featuremaps obtainedfrom the first dense block is 96, and a subsequent growth rate is 48;the size of the output feature maps of the feature extraction layer isof ⅛ input image size, that is 56×56; the second part of the denselyconnected dilated convolutional neural network is three denselyconnected dilated convolutional layers, of which the dilation rates are3, 6 and 12 respectively; the input of each dilated convolutional layeris the output of all previous dilated convolutional layers; the featuremap size of final output of the densely connected dilated convolutionalnetwork is 56×56.

in the end of the network is an upper sampling layer with an uppersampling rate of 8 and a classification layer to predict the probabilityof belonging to pancreas region for each pixel in the image; the lossfunction of the network is set as:

$E = {{- \frac{1}{n}}{\sum\limits_{j = 1}^{n}{\sum\limits_{c \in {\{{0,1}\}}}{{I\left( {y_{j} = c} \right)}\log\mspace{14mu}{p\left( {z_{j} = c} \right)}}}}}$

where n is the number of pixels in the input image, y_(j) and z_(j) aretrue label and predicted label of pixel j, respectively, and c=1 or 0represents the foreground or the background, respectively. The function1(·) is a characteristic function, the function log is a logarithmicfunction, and p(·) is a probability function predicted by the networkmodel. The symbol Σ is a summation symbol, and the symbol E representsbelonging.

(2.2) training parameters of the densely connected dilated neuralconvolutional network with the training set samples; feeding a preparedtraining set into the network, and obtaining network parameters by theback-propagation algorithm; obtaining a pancreas coarse segmentationmodel after training.

(2.3) feeding the test image into the trained network model to assignthe probability value P_(A,l) ^(C) belonging to the pancreas for eachpixel in the image; binarizing the probability value to obtain aninitial segmentation result Z_(l) ^(C) of the pancreas, where thebinarization threshold is set to be 0.5.

(2.4) cropping a region of interest on the original input imageaccording to the initial segmentation result. Specifically, a boundingbox of Z_(l) ^(C) is calculated, and the bounding box is padded by m,the m is set to 20 pixels. According to the bounding box, the originalimage is cropped and denoted as X_(A,l) ^(F). Similarly, the originallabel image is cropped and denoted as Y_(A,l) ^(F).

(3) calculating a saliency map based on geodesic distance

(3.1) calculating a geodesic distance map with the original imageX_(A,l) ^(F) and the initial segmentation result Z_(l) ^(C); denotingsets of pixels belonging to the pancreas area and the background asS_(f) and S_(g), respectively according to the initial segmentationZ_(l) ^(C); the geodesic distance G⁰(i,S_(f),X_(A,l) ^(F)) from pixel ito S_(f) is defined as:

$\begin{matrix}{{{G^{0}\left( {i,S_{f},X_{A,l}^{F}} \right)} = {{{}_{j \in S_{f}}^{}{}_{}^{}}\left( {i,j,X_{A,l}^{F}} \right)}},{{D_{geo}\left( {i,j,X_{A,l}^{F}} \right)} = {\min\limits_{p \in {\mathcal{P}{({i,j})}}}{\int_{0}^{1}{{{{\nabla{X_{A,l}^{F}\left( {p(s)} \right)}} \cdot {u(s)}}}d\; s}}}},} & \;\end{matrix}$

wherein

(i, j) is a set of all feasible paths from pixel i to j; path p isparameterized by s∈[0,1] as p(s);

${u(s)} = \frac{{p(s)}^{\prime}}{{p(s)}^{\prime}}$

is a unit vector that is tangent to the the path direction, and p(s)′represents derivation for s; the image derivative ∇X_(A,l) ^(F)(p(s))requires the path from i to j to be the shortest in terms of imagegray-scale similarity; the symbol ∫₀ ¹ ds represents an integral of sfrom 0 to 1, the symbol ∥ ∥ represents the l₁ norm, and the symbol ∈represents belonging to.

We set the geodesic distance map as G(S_(f),X_(A,l) ^(F)), and thecalculation is as follows:

${G\left( {i,S_{f},X_{A,l}^{F}} \right)} = {1 - \frac{G^{0}\left( {i,S_{f},X_{A,l}^{F}} \right)}{\max\limits_{i}\;{G^{0}\left( {i,S_{f},X_{A,l}^{F}} \right)}}}$

where i is a pixel in the geodesic distance map.

(3.2) performing saliency transformation on the geodesic distance map toobtain a saliency map S(X_(A,l) ^(F)):

S(X _(A,l) ^(F))=r(G(S _(f) ,X _(A,l) ^(F)))

where r(·) is a size-preserved saliency transformation that uses one 3×3convolutional layer.

(4) performing the fine segmentation on the pancreas CT by asaliency-aware densely connected dilated convolutional neural network

(4.1) introducing a saliency-aware module into the densely connecteddilated convolutional neural network to construct the saliency-awaredensely connected dilated convolutional neural network; adding thesaliency-aware module after each dense block of the densely connecteddilated convolutional neural network constructed by step (2) tointroduce saliency information based on the geodesic distance map;specifically, denoting F(X_(A,l) ^(F)) as a output feature map of thedense block, and taking the saliency map S(X_(A,l) ^(F)) as a weight mapto act on the feature map:

L_(c)(X _(A,l) ^(F))=F _(c)(X _(A,l) ^(F))└S(X _(A,l) ^(F)),

where c∈{1, 2, . . . , C} is the index of a feature map channel, and └represents element-wise multiplication; further, combining the obtainedL_(c)(X_(A,l) ^(F)) with the original feature map F_(c)(X_(A,l) ^(F))through an identity transformation and a parameter η:

H_(c)(X _(A,l) ^(F))=F_(c)(X _(A,l) ^(F))└η*L _(c)(X _(A,l) ^(F)),

where the symbol └ represents element-wise addition, and the parameter ηis an adjustment coefficient, which can be obtained through networktraining; the output H_(c)(X_(A,l) ^(F)) of the obtained saliency-awaremodule is used as an input of a next transition layer to participate incalculation.

(4.2) feeding the original image X_(A,l) ^(F) and the correspondinglabel image Y_(A,l) ^(F) into the constructed saliency-aware denselyconnected dilated convolutional neural network for parameter training,and obtaining a fine segmentation network model.

(5) fusing multi-layer two-dimensional pancreatic segmentation resultsto obtain a three-dimensional pancreatic segmentation result

(5.1) for a test image X^(test) , slicing X^(test) along axial view toobtain a two-dimensional image sequence, and combining three consecutivegray-scale images into a three-channel pseudo-color image X_(A,l)^(F)(l=1, . . . , L); feeding each image into the trained coarsesegmentation model and the fine segmentation model successively, andobtaining a prediction result P_(A,l) for each image about pancreasareas. P_(A,l) is a three-channel image, and its first, second and thirdchannels correspond to the probability values of the pancreas at thel=1, and l+1slices of the original CT volume, respectively.

(5.1) performing multi-layer probability value fusion on the predictionresult P_(A,l) , and a predicted segmentation result Z_(A)(l) of eachtwo-dimensional original image is an average value of predicted valuesof three pseudo-color images in this layer, that is:

Z _(A)(l)=⅓(P _(A,l−1)(3)+P _(A,l)(2)+P_(A,l+1)(1)),

wherein, P_(A,l)(i),i=1, 2, 3 represent the value of an i^(th) channelof P_(A,l) , respectively

We tested our model on 82 pancreatic CT data. All data is divided intofour parts with cross-validation method, numbered 1, 2, 3, and 4. In thefirst experiment, numbers 2, 3, 4 are for training, 1 is for testing. Inthe second experiment, 1, 3, 4 are for training, 2 is for testing, andso on. Finally, the accuracy of the four sets of experiments isaveraged. Experiments show that the fine segmentation accuracy using theDenseNet161 network model is 82.83%, the segmentation accuracy using thedensely connected dilated convolutional neural network model is 83.88%,and the accuracy using the saliency-aware densely connected dilatedconvolutional neural network model of the present invention is 85.31%.This shows that the densely connected dilated convolutional network andthe saliency-aware module used and proposed in the present invention caneffectively improve the pancreatic segmentation accuracy.

FIG. 3 shows the segmentation results of our proposed saliency-awaredensely connected dilated convolutional neural network model ondifferent layers of one CT volume. The white contours are thealgorithmic segmentation results, and the black contours are the goldstandard. The coarse segmentation accuracy is 80.27%, and the finesegmentation accuracy is 85.23%.

The inventive point of the present invention is: the present inventionuses densely connected dilated convolutional neural network to segmentthe pancreas CT to obtain a densely multi-scale feature representationof the pancreas; based on the geodesic distance transformation and thesingle-layer convolutional layer transformation, the saliencytransformation is performed on the initial segmentation result to obtainthe saliency information about the pancreas; the saliency-aware moduleis introduced to the densely connected dilated network, and thesaliency-aware densely connected dilated convolutional neural network isconstructed, so that the network obtains the saliency information aboutthe pancreatic region in the feature extraction stage.

The above are only the preferred embodiments of the present invention.Although the present invention has been disclosed as above in preferredembodiments, it is not intended to limit the present invention. Anyonefamiliar with the art, without departing from the scope of the technicalsolution of the present invention, can use the methods and technicalcontent disclosed above to make many possible changes and modificationsto the technical solution of the present invention, or modify itmodified into an equivalent embodiment with equivalent changes. Forexample, the densely connected dilated convolutional neural network inthe present invention can also be replaced with other deep convolutionalneural network models, such as U-net, DenseNet, ResNet and so on. Thereplacement of the network models does not affect the introduction ofthe saliency-aware module. The technical solution of the presentinvention is used for pancreas CT data, the imaging modality can also bereplaced with other imaging data such as magnetic resonance imaging(MRI), and the segmented target pancreas can be replaced with otherrelative small organs or tissues such as the gallbladder. Therefore, allsimple modifications, equivalent changes and modifications made to theabove embodiments based on the technical essence of the presentinvention without departing from the technical solution of the presentinvention still fall within the protection scope of the technicalsolution of the present invention.

What is claimed is:
 1. A pancreas CT automatic segmentation method basedon a saliency-aware densely connected dilated convolutional neuralnetwork, comprising the following steps of: (1) preprocessing oftraining set, comprising the following steps of: collecting CT volumedata and making a standard pancreas segmentation result of the data;denoting 3D CT volume data as X, and slice number of the volume data asL, a corresponding standard segmentation being Y=(y_(j),j=1, . . . ,|X|),y_(j)={0,1}, where |X| represents a number of all voxels in X,y_(j)=1 or y_(j)=0 represents that voxel j belongs to the pancreas or abackground, respectively; Slicing each volume X into two-dimensionalimage slices alone axial view; and combining three consecutive imagesinto a three-channel pseudo-color image, denoted as X_(A,l)(l=1, . . . ,L) Slicing Y into two-dimensional image slices alone axial view, andcombining three consecutive label images into a three-channel labelimage, denoted as Y_(A,l)(l=1, . . . , L); adjusting a contrast of eachtwo-dimensional image X_(A,l); cropping each pair of two-dimensionalimages X_(A,l) and Y_(A,l) into fixed size image block as input of acoarse segmentation network; (2) coarse segmentation stage, comprisingthe following steps of: constructing a deep convolutional neural networkfor coarse segmentation for pancreas CT, and training the network bytraining samples to obtain a pancreas initial segmentation model;feeding the test image into the trained network model to assign eachpixelin the image to be assigned a probability value P_(A,l) ^(C) ofbelonging to the pancreas; binarizing the probability value to obtain aninitial segmentation result Z_(l) ^(C) of the pancreas; cropping theregion of interest on the original input image according to the initialsegmentation result, and denoting the cropped images as X_(A,l) ^(F);Similarly, cropping the label image and denoting it as Y_(A,l) ^(F). (3)calculating a saliency map based on a geodesic distance, comprising:calculating a geodesic distance map according to the original imageX_(A,l) ^(F) and the initial segmentation result Z_(l) ^(C); performinga saliency transformation on the geodesic distance map to obtain asaliency map S(X_(A,l) ^(F)); (4) fine segmentation stage, comprisingthe following steps of: constructing a saliency-aware densely connecteddilated convolutional neural network: adding a saliency-aware moduleafter each dense block of the densely connected dilated convolutionalneural network to introduce saliency information based on the geodesicdistance map. Denoting F(X_(A,l) ^(F)) as output feature map of thedense block, and taking the saliency map S(X_(A,l) ^(F)) as a weight mapto act on the feature map:L _(c)(X _(A,l) ^(F))=F _(c)(X _(A,l) ^(F))└S(X _(A,l) ^(F)), wherec∈{1, 2, . . . , C} is an index of feature map channel, and └ representselement-wise multiplication; combining the obtained L_(c)(X_(A,l) ^(F))with the original feature map F_(c)(X_(A,l) ^(F)) through an identitytransformation and a parameter η:H _(c)(X _(A,l) ^(F))=F _(c)(X _(A,l) ^(F))└η*L _(c)(X _(A,l) ^(F)),where the symbol └ represents element-wise addition, and the parameter ηis an adjustment coefficient, which is obtained through networktraining. The output H_(c)(X_(A,l) ^(F)) of the obtained saliency-awaremodule is used as an input of next transition layer to participate inthe calculation; feeding the original image X_(A,l) ^(F) and thecorresponding label image Y_(A,l) ^(F) into the constructedsaliency-aware densely connected dilated convolutional neural networkfor parameter training, and obtaining a fine segmentation network model;(5) fusing multi-layer two-dimensional pancreatic segmentation resultsto obtain a three-dimensional pancreatic segmentation result, for a testimage X^(test), slicing X^(test) along an axial view to obtain atwo-dimensional image sequence, and combining three consecutivegray-scale images into a three-channel pseudo-color image X_(A,l)^(test)(l=1, . . . , L) ; feeding each image into the trained coarsesegmentation model and the fine segmentation model successively, andobtaining a prediction result P_(A,l) for each image about pancreasarea; performing multi-layer probability value fusion on the predictionresult P_(A,l) , and a predicted segmentation result Z_(A)(l) of thetwo-dimensional original image of each layer is the average value ofpredicted values of three pseudo-color images in this layer.
 2. Theautomatic pancreas CT segmentation method based on a saliency-awaredensely connected dilated convolutional neural network according toclaim 1, where the step of adjusting the contrast of eachtwo-dimensional image X_(A,l) is specifically: the HU values of imagesare truncated into range 00, 240], and then normalized to be in range[0, 1].
 3. The automatic pancreas CT segmentation method based on asaliency-aware densely connected dilated convolutional neural networkaccording to claim 1, wherein in the coarse segmentation stage, thedensely connected dilated convolutional network is configured to performcoarse segmentation on pancreas CT, and the network is composed of twoparts: a first part is a feature extraction layer of a densely connectednetwork 161, comprising a convolution-normalization-ReLUactivation-pooling layer, four dense blocks and four transition layers;the number of feature maps obtained from a first dense block is 96, anda subsequent growth rate is 48; the size of the output feature map ofthe feature extraction layer is of ⅛ input image size; a second part isthree densely connected dilated convolutional layers, dilation rate are3, 6 and 12, respectively; and an input of each dilated convolutionallayer is the output of all previous dilated convolutional layers; in theend of the network is a network connects an upper sampling layer with anupper sampling rate of 8 and a classification layer to predict theprobability of belonging to pancreas region for each pixel in the image;the loss function of the network is set as:$E = {{- \frac{1}{n}}{\sum\limits_{j = 1}^{n}{\sum\limits_{c \in {\{{0,1}\}}}{{I\left( {y_{j} = c} \right)}\log\mspace{14mu}{p\left( {z_{j} = c} \right)}}}}}$where, n is the number of pixels in the input image, y_(j) and z_(j) aretrue label and predicted label of pixel j, respectively, and c=1 or 0represents the foreground or the background, respectively, the function1(·) is an characteristic function, and p(·) is a probability functionpredicted by the network model.
 4. The automatic pancreas CTsegmentation method based on a saliency-aware densely connected dilatedconvolutional neural network according to claim 3, where in the coarsesegmentation stage, feeding a prepared training set into the denselyconnected dilated convolutional neural network, and obtaining networkparameters by the back-propagation algorithm.
 5. The automatic pancreasCT segmentation method based on a saliency aware densely connecteddilated convolutional neural network according to claim 1, where thegeodesic distance map G(S_(f),X_(A,l) ^(F)) is specifically calculatedas follows: denoting sets of pixels belonging to the pancreas area andthe background as S_(f) and S_(g), respectively with initialsegmentation Z_(l) ^(C); the geodesic distance G⁰(i,S_(f),X_(A,l)^(F))from a pixel i to S_(f) is defined as: $\begin{matrix}{{{G^{0}\left( {i,S_{f},X_{A,l}^{F}} \right)} = {{{}_{j \in S_{f}}^{}{}_{}^{}}\left( {i,j,X_{A,l}^{F}} \right)}},{{D_{geo}\left( {i,j,X_{A,l}^{F}} \right)} = {\min\limits_{p \in {\mathcal{P}{({i,j})}}}{\int_{0}^{1}{{{{\nabla{X_{A,l}^{F}\left( {p(s)} \right)}} \cdot {u(s)}}}d\; s}}}},} & \;\end{matrix}$ where

(i,j) is a set of all feasible paths from pixel i to j; a path p isparameterized by s∈[0,1] as p(s);${u(s)} = \frac{{p(s)}^{\prime}}{{p(s)}^{\prime}}$ is a unit vectorthat is tangent to the path direction, and p(s)′ represents derivationfor s; an image derivative ∇X_(A,l) ^(F)(p(s)) requires the path from ito j to be the shortest in terms of image gray-scale similarity;denoting the geodesic distance map as G(S_(f),X_(A,l) ^(F)), and thecalculation is as follows:${G\left( {i,S_{f},X_{A,l}^{F}} \right)} = {1 - \frac{G^{0}\left( {i,S_{f},X_{A,l}^{F}} \right)}{\max\limits_{i}\;{G^{0}\left( {i,S_{f},X_{A,l}^{F}} \right)}}}$where i is a pixel in the geodesic distance map.
 6. The automaticpancreas CT segmentation method based on a saliency-aware denselyconnected dilated convolutional neural network according to claim 5,where the saliency map S(X_(A,l) ^(F)) is calculated as follows:S(X _(A,l) ^(F))=r(G(S _(f) ,X _(A,l) ^(F))) where r(·) is asize-preserved saliency transformation that uses one 3×3 convolutionallayer.
 7. The automatic pancreas CT automatic segmentation method basedon a saliency-aware densely connected dilated convolutional neuralnetwork according to claim 1, where in step (5), the predictedsegmentation result Z_(A)(l) of the two-dimensional original image ofeach layer is calculated as follows:Z _(A)(l)=⅓(P _(A,l−1)(3)+P _(A,l)(2)+P _(A,l+1)(1)); whereP_(A,l)(i),i=1, 2, 3 represent P_(A,l) value of an i^(th) channel,respectively.
 8. An automatic pancreas CT segmentation system based on asaliency-aware densely connected dilated convolutional neural network,comprising: a memory for storing computer executable instructions; and aprocessor for realizing the steps in the method according to claim 1when executing the computer executable instructions.
 9. Acomputer-readable storage medium having stored therein computerexecutable instructions which implement the steps in the method of claim1 when executed by a processor.